Assessing the accuracy of predictive models for numerical data: Not r nor r2, why not? Then what?
نویسنده
چکیده
Assessing the accuracy of predictive models is critical because predictive models have been increasingly used across various disciplines and predictive accuracy determines the quality of resultant predictions. Pearson product-moment correlation coefficient (r) and the coefficient of determination (r2) are among the most widely used measures for assessing predictive models for numerical data, although they are argued to be biased, insufficient and misleading. In this study, geometrical graphs were used to illustrate what were used in the calculation of r and r2 and simulations were used to demonstrate the behaviour of r and r2 and to compare three accuracy measures under various scenarios. Relevant confusions about r and r2, has been clarified. The calculation of r and r2 is not based on the differences between the predicted and observed values. The existing error measures suffer various limitations and are unable to tell the accuracy. Variance explained by predictive models based on cross-validation (VEcv) is free of these limitations and is a reliable accuracy measure. Legates and McCabe's efficiency (E1) is also an alternative accuracy measure. The r and r2 do not measure the accuracy and are incorrect accuracy measures. The existing error measures suffer limitations. VEcv and E1 are recommended for assessing the accuracy. The applications of these accuracy measures would encourage accuracy-improved predictive models to be developed to generate predictions for evidence-informed decision-making.
منابع مشابه
Evaluation of an evaluation
Introduction. Evaluation is a systematic way to improve and make more effective actions that involves procedures which are useful, feasible, ethical, and accurate. Common questions in all evaluations are: do all part of program do well and effective? What is the good functioning? Why the program or its parts do not work well? What are the effects and consequences of the program? Is this progra...
متن کاملطراحی شبکه عصبی مصنوعی برای مدلبندی پاسخهای دو متغیره آمیخته و کاربرد آن در دادههای پزشکی
Background & Objective: Mixed outcomes arise when, in a multivariate model, response variables measured on different scales such as binary and continuous. Artificial neural networks (ANN) can be used for modeling in situations where classic models have restricted application when some of their assumptions are not met. In this paper, we propose a method based on ANNs for modeling mixed binary a...
متن کاملComparison of Efficiency for Hydrological Models (AWBM & SimHyd) and Neural Network (MLP & RBF) in Rainfall–Runoff Simulation (Case study: Bar Aryeh Watershed -Neyshabur)
For suitable programming and management of water resources, access to perfect information from the discharge at the watershed outlet is essential. In most watersheds, the hydrometric station is not available; then, different models are used to simulate the discharge within watersheds without data. The selection of preferred model for rainfall- runoff simulation depends to the purpose of modelin...
متن کاملAn Empirical Study about Why Dissatisfaction Arises Among the Employees and What It Consequences: Bangladesh Perspective
This article aimed at identifying the rate of dissatisfied employees who had left their previous jobs and the main factors which caused their dissatisfaction. In order to collect data for this study a well-structured questionnaire was distributed to 150 employees of different private and public organizations in Bangladesh who already left their previous jobs and 142 usable responses were rec...
متن کاملSpatiotemporal Estimation of PM2.5 Concentration Using Remotely Sensed Data, Machine Learning, and Optimization Algorithms
PM 2.5 (particles <2.5 μm in aerodynamic diameter) can be measured by ground station data in urban areas, but the number of these stations and their geographical coverage is limited. Therefore, these data are not adequate for calculating concentrations of Pm2.5 over a large urban area. This study aims to use Aerosol Optical Depth (AOD) satellite images and meteorological data from 2014 to 2017 ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره 12 شماره
صفحات -
تاریخ انتشار 2017